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(54) Motion estimation apparatus and method 

(57) A motion estimation system, particularly for tel- 
evision applications, comprises a 3D-recursive search 
block-matching device for generating motion vectors 
representing the motion of the content of pixel blocks in 
response to successive fields or frames of video signals. 
This block-matching device is provided with a motion es- 
timator (2) for generating estimated motion vectors, a 
spatial prediction memory (3) and/or a temporal predic- 
tion memory (4), wherein the estimated motion vectors 
are stored. The stored motion vectors form a basis for 
generating further motion vectors for next pixel blocks 
and/or next images. Between the spatial prediction 
memory (3) and/or the temporal prediction memory (4) 
on the one side and the motion estimator (2) on the other 
side a vector filter (5,6) is provided, which, when a pre- 
diction candidate is coming from a location outside a 
predefined neighborhood, either substitutes the predic- 
tion candidate with a safe one or modifies the prediction 
candidate according to a predefined algorithm. 
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Description 

[0001] The present invention relates to a motion esti- 
mation, particularly for television applications, with a3D- 
recursive search block-matching for generating motion 
vectors representing the motion of the content of pixel 
blocks in response to successive fields or frames of vid- 
eo signals, the 3D-recursive search block-matching 
comprising a motion estimation for generating estimat- 
ed motion vectors, a spatial prediction memory and/or 
a temporal prediction memory, in which memories the 
estimated motion vectors are stored, which stored mo- 
tion vectors, called prediction candidates, form a basis 
for generating further motion vectors for next pixel 
blocks and/or next images. 

[0002] The prediction candidates could be taken from 
spatial and/or temporal neighborhoods of a current es- 
timation position. The motion estimator contains a 
matching criterion to select the best of these prediction 
candidates and other candidates including those de- 
rived from prediction vectors through addition, for exam- 
ple, of random (update) vectors. The motion vectors ob- 
tained are used for motion compensated processing, 
such as field-rate up-conversion, de-interlacing or noise 
reduction. 

[0003] A motion estimation system as described 
above, is known from De Haan et al.: "True-motion es- 
timation"; IEEE Transactions on circuits and systems for 
video technology, Vol. 3, No. 5, October 1993, p. 
368-379. 

[0004] A problem with such an estimation process is 
that, when, in case of pixel blocks located near the bor- 
der of the active area of the video signals, prediction 
candidates are necessarily taken from outside the active 
area, i.e. outside the useful image area, they can have 
a value that has no relation with the video content of the 
pixel blocks. Particularly, when the video image contains 
periodic structures, where wrong motion vectors can 
match equally well or even better than true motion vec- 
tors, they introduce a risk of wrong, wild or noisy motion 
vectors, leading to objectionable processing artifacts. 
[0005] An object of the invention is to provide an im- 
proved motion estimation. To this end, the invention pro- 
vides a motion estimation as defined in the independent 
claims. Advantageous embodiments are defined in the 
dependent claims. 

[0006] According to one aspect of the invention, the 
motion estimation system as described in the opening 
paragraph is characterized in that between the spatial 
prediction memory and/or the temporal prediction mem- 
ory on the one side and the motion estimator on the oth- 
er side a vector filter is provided, which, when a predic- 
tion candidate is coming from a location outside a pre- 
defined safe neighborhood, either substitutes the pre- 
diction candidate with a safe one or modifies the predic- 
tion candidate according to a predefined algorithm. The 
result is that random candidate vectors are prevented 
from participating in the motion estimation process. The 



safe neighborhood is coupled to the useful image areas 
to adapt the motion vector filter to changing geometry. 
[0007] Instead of the above preventive measure or to- 
gether therewith a curative measure is possible. In that 

5 case the output signal of the motion estimator is sup- 
plied to a vector filter, which, when an estimated motion 
vector is based on a prediction candidate coming from 
a location outside a predefined safe neighborhood, the 
filter either substitutes the estimated motion vector with 

10 a safe one or modifies the estimated motion vector ac- 
cording to a predefined algorithm. Subsequent motion 
compensated processing no longer suffers from spuri- 
ous motion vectors. 

[0008] These and other aspects of the invention will 
15 be apparent from and elucidated with reference to the 
embodiments described hereinafter. 
[0009] The invention will further be described with ref- 
erence to the drawing, which shows a recursive search 
block-matching device for a motion estimation system 
20 according to the invention. 

[0010] The drawing shows a 3D-recursive search 
block-matching device for generating motion vectors. 
These motion vectors represent the motion of the con- 
tent of pixel blocks in response to successive fields or 
25 frames of video signals. The recursive block-matching 
device 1 comprises a motion estimator 2 for generating 
estimated motion vectors, a spatial prediction memory 

3 and a temporal prediction memory 4, in which memo- 
ries the estimated motion vectors are stored. These 

30 stored motion vectors are the prediction candidates for 
generating further motion vectors for next pixel blocks 
and/or next images. 

[0011] Contrary to full search block-matching, in 
which each block of pixels in a search area is compared 
35 with each block in a former image, in the recursive 
search block-matching device, a pixel block is only com- 
pared with a limited number of blocks. In first instance, 
this results in a rather coarse motion vector estimation. 
By a repeated comparison with spatial or temporal 
40 neighboring pixel blocks, each corresponding with a mo- 
tion vector (candidate), from which can be expected that 
they have some relation with the block corresponding 
with the mentioned coarsely estimated motion vector 
(candidate), a more accurate motion vector can be es- 
45 tablished, which again can be considered as a candi- 
date for a further establishing of the motion vector. 
[0012] In a first embodiment, between the spatial pre- 
diction memory 3 and the temporal prediction memory 

4 on the one side, and the motion estimator 2 on the 
so other side, a vector filter 5 and 6 respectively is provided. 

When a prediction candidate is coming from a location 
outside a predefined safe area, such as information from 
an invalid part of the display or from a part of the display 
which has valid information not being relevant for mo- 
ss tion, the filter either substitutes the prediction candidate 
with a safe one or modifies the prediction candidate ac- 
cording to a predefined algorithm. Particularly such a 
candidate can be replaced by a zero-vector. 
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[0013] To replace or modify prediction candidates 
they must be identified as corresponding with a pixel 
block outside the useful video area. Therefore, in a sec- 
ond embodiment, the co-ordinates of the useful video 
area are supplied as parameters to the motion estimator 
2, and the vectors outside these co-ordinates are e.g. 
set to a value of zero magnitude. In an another embod- 
iment the prediction vectors outside the predefined area 
are assigned a constant magnitude that will point to a 
predefined data area in the video image. In still another 
embodiment the prediction vectors that come from out- 
side the predefined area are chosen only if on evaluation 
they lead to match errors that are lower than those of all 
other prediction candidates from within the said prede- 
fined area by a predefined minimum threshold. 
[0014] Instead of or besides the vector filters 5 and 6, 
a vector filter 7 can be provided at the output of the mo- 
tion estimator 2. By means of this vector filter 7 vectors 
from outside the useful video area can be given a pre- 
determined safe value or modified otherwise. The out- 
put signals of the motion estimator 2 are directly or via 
the vector filter 7 supplied to a usual unit 8 for motion- 
compensated processing such as a motion-compensat- 
ed field rate doubling. An output of the motion-compen- 
sated processing unit 8 is applied to a display device 9. 
[0015] It should be noted that the above-mentioned 
embodiments illustrate rather than limit the invention, 
and that those skilled in the art will be able to design 
many alternative embodiments without departing from 
the scope of the appended claims. In the claims, any 
reference signs placed between parentheses shall not 
be construed as limiting the claim. The word "compris- 
ing" does not exclude the presence of elements or steps 
other than those listed in a claim. The word "a" or "an" 
preceding an element does not exclude the presence of 
a plurality of such elements. The invention can be im- 
plemented by means of hardware comprising several 
distinct elements, and by means of a suitably pro- 
grammed computer. In the device claim enumerating 
several means, several of these means can be embod- 
ied by one and the same item of hardware. The mere 
fact that certain measures are recited in mutually differ- 
ent dependent claims does not indicate that a combina- 
tion of these measures cannot be used to advantage. 



Claims 

1. A motion estimation system, particularly for televi- 
sion applications, with a recursive search block- 
matching device for generating motion vectors rep- 
resenting the motion of the content of pixel blocks 
in response to successive fields or frames of video 
signals, the recursive search block-matching de- 
vice comprising a motion estimator (2) for generat- 
ing estimated motion vectors, a spatial prediction 
memory (3) and/or a temporal prediction memory 
(4), in which memories (3,4) the estimated motion 



vectors are stored, which stored motion vectors, 
called prediction candidates, form a basis for gen- 
erating further motion vectors for next pixel blocks 
and/or next images, characterized in that between 

5 the spatial prediction memory (3) and/or the tempo- 
ral prediction memory (4) on the one side and the 
motion estimator (2) on the other side a vector filter 
(5,6) is provided, which, when a prediction candi- 
date is coming from a location outside a predefined 

10 safe area, either substitutes the prediction candi- 
date with a safe one or modifies the prediction can- 
didate according to a predefined algorithm. 

2. A motion estimation system according to claim 1, 
15 characterized in that the output signal of the motion 

estimator (2) is supplied to a vector filter (7), which, 
when an estimated motion vector is based on a pre- 
diction candidate coming from a location outside a 
predefined safe area, either substitutes the estimat- 
20 ed motion vector with a safe one or modifies the es- 
timated motion vector according to a predefined al- 
gorithm. 

3. A motion estimation system, particularly for televi- 
25 sion applications, with a recursive search block- 
matching device for generating motion vectors rep- 
resenting the motion of the content of pixel blocks 
in response to successive fields or frames of video 
signals, recursive search block-matching device 

30 comprising a motion estimator (2) for generating es- 
timated motion vectors, a spatial prediction memory 
(3) and/or a temporal prediction memory (4), in 
which memories (3,4) the estimated motion vectors 
are stored, which stored motion vectors, called pre- 

35 diction candidates, form a basis for generating fur- 
ther motion vectors for next pixel blocks and/or next 
images, characterized in that the output signal of 
the motion estimator (2) is supplied to a vector filter 
(7), which, when an estimated motion vector is 

40 based on a prediction candidate coming from a lo- 
cation outside a predefined safe area, either substi- 
tutes the estimated motion vector with a safe one 
or modifies the estimated motion vector according 
to a predefined algorithm. 

45 

4. A motion estimation system according to any one 
of the preceding claims, characterized in that the 
co-ordinates of the predefined safe area are sup- 
plied as parameters to the motion estimator (2), and 

so the vectors outside these co-ordinates are set to a 
value of zero magnitude. 

5. A motion estimation system according to any one 
of the claims 1-3, characterized in that prediction 

55 vectors outside the predefined area are assigned a 
constant magnitude that will point to a predefined 
data area in the video image. 
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6. A motion estimation system according to any one 
of the claims 1-3, characterized in that prediction 
vectors that come from outside the predefined safe 
area are chosen only if on evaluation they lead to 
match errors that are lower than those of all other 5 
prediction candidates from within the said prede- 
fined safe area by a predefined minimum threshold. 

7. A motion estimation method, comprising the steps 



estimating (2) motion vectors; 
storing (3,4) the motion vectors to obtain pre- 
diction vectors; 

applying (5, 6) the prediction vectors to the es- 15 
timating step (2); and 

processing (2, 5, 6, 7) the prediction vectors or 
the motion vectors for modifying or replacing 
the prediction vectors or the motion vectors 
when the prediction vectors come from a loca- 2 o 
tion outside a predefined safe area. 



8. A motion estimation device, comprising: 



means for estimating (2) motion vectors; 

means for storing (3,4) the motion vectors to 

obtain prediction vectors; 

means for applying (5, 6) the prediction vectors 

to the estimating step (2); and 

means for processing (2, 5, 6, 7) the prediction 

vectors or the motion vectors for modifying or 

replacing the prediction vectors or the motion 

vectors when the prediction vectors come from 

a location outside a predefined safe area. 



9. A television apparatus, comprising: 



a motion estimation device (2-7) as claimed in 
claim 8 for generating motion vectors; 
a motion-compensated processing unit (8) for 40 
processing a video signal in dependence upon 
the motion vectors to obtain a processed video 
signal; and 

a display device (9) for displaying the proc- 
essed video signal. 45 
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